From Accountability to Prevention: 

Early Warning Systems Put Data to Work 
for Struggling Students 

May 2015 


Mindee O’Cummings and Susan Bowles Therriault 


EARLY WARNING SYSTEMS 
in EDUCATION 

College & Career Readiness 

at American Institutes for Research ■ 

1000 Thomas Jefferson Street NW 
Washington, DC 20007-3835 
202-403-5000 I TTY 877-334-3499 
www.earlywarningsystems.org 

Copyright © 2015 American Institutes for Research. All rights reserved. 

HAIR 

American Institutes for Research* 


www.air.org 


2298_05/15 




Contents 


Page 

Introduction.1 

About Early Warning Systems.2 

School-Level Lessons Learned.4 

Empower Action Through Effective Leadership.4 

Beware of Overwhelming Staff.5 

Take a Whole-Student, Whole-School Approach.6 

District-Level Lessons Learned.8 

Empower Schools.8 

Identify and Address Districtwide Needs.8 

Streamline Access to Data.8 

State-Level Lessons Learned.9 

Help Districts and Schools Define and Validate Indicators.9 

Develop Customized Supports.10 

Use Levers to Encourage and Support Implementation.11 

Conclusion.13 


References 


14 



















Introduction 


Educators at all levels care deeply about helping students succeed academically, graduate on 
time, and emerge from school well prepared for college and careers. Today’s emphasis on 
holding schools accountable for failure—while necessary—is at best a means to an end. Findings 
from local and statewide accountability systems can help state education agencies (SEAs) direct 
funding and other resources to the lowest performing schools, where needs are great. However, 
struggling students exist in all schools and districts, not only those that fall short of performance 
thresholds. Schools need ways to identify these students long before their difficulties derail their 
hopes for graduation and beyond. More state, district, and school leaders are now realizing that 
the same data they collect for accountability measures also can be used to improve instruction 
and support better outcomes for students. Early warning systems offer a way to use available 
data to identify students at risk and drive targeted interventions that help them get back on track. 

Early warning system research originated in districts such as Chicago and Philadelphia (e.g., 
Allensworth & Easton, 2005; Allens worth & Easton, 2007; Balfanz & Herzog, 2005; Neild & 
Balfanz, 2006; Neild, Balfanz & Herzog, 2007). However, more than a decade of student-level 
data collection has prompted growing interest in early warning systems among state 
policymakers and leaders as well. Several states, such as California, Hawaii, Massachusetts, 
Virginia, and Texas, are already pioneering efforts to build and implement early warning 
systems. According to a recent survey of SEAs, as many as 27 other states 1 are poised to do the 
same (Data Quality Campaign, 2013). In each of these states, the approach has been multifaceted 
and emphasizes the engagement of districts and schools—where actions can be taken directly to 
support students. Early warning systems can be implemented by schools alone; however, districts 
and states have important roles to play in supporting schools and increasing the effectiveness of 
their efforts (Heppen & Therriault, 2008). 

The intent of this Issue Paper is to support leaders in schools, districts, and states who are 
considering developing, implementing, or refining early warning systems. American Institutes 
for Research (AIR) has provided targeted early warning system implementation support to more 
than 300 schools, 100 districts, and 12 states and has offered technical assistance to countless 
others. Our experts have found that although certain recommendations are applicable specifically 
to leaders at the school, district, or state level, a broad understanding of the needs and challenges 
at each level enhances implementation overall. The organization of this brief thus addresses each 
of these levels, beginning with schools, and is based on AIR’s lessons learned from more than 
seven years of working on early warning systems with schools, districts, and states. 


1 Data Quality Campaign reports that a total of 31 states produce early warning system reports. Hawaii, 
Massachusetts, Texas, and Virginia are all included among this count. California does not provide an early warning 
system report but supports implementation of early warning systems in the middle grades through district support. 
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About Early Warning Systems 

Early warning systems use readily available data to identify students at risk of missing key 
educational milestones, to diagnose their needs, and to provide supports and interventions that 
help get students back on track. Early warning data can be examined to better understand the 
needs of individual students, groups of students, or the school as a whole. 

The use of early warning systems to prevent high school dropout is a recommended evidence- 
based strategy (Dynarski et al., 2008). Recently, some SEAs, such as Massachusetts (Therriault 
& Jung, 2013), have broadened the notion of early warning systems to identify students who are 
at risk of missing other key academic milestones such as reading by the end of third grade, being 
middle school ready, or being college ready. 


Key Terms 

Early warning systems are data systems that use validated indicators to identify students 
at risk of missing key educational milestones such as high school graduation. 

AIR’s Early Warning Intervention and Monitoring System (EWIMS) is a seven-step 
process to support the establishment and implementation of early warning systems. 
EWIMS provides practical guidance on how to make data actionable, with the aim of 
improving outcomes for students. 

The Early Warning System (EWS) Tool, which can be used as part of the EWIMS 
process, provides the means by which schools and districts can identify at-risk students 
and monitor these students’ responses to interventions. Outcome measures, indicators, and 
thresholds (see below) are predefined within the tool. After student data are loaded into the 
tool, its reports can be used to make decisions about how to support at-risk students as 
individuals, in small groups, and within the school as a whole. 

Outcome measures are future milestone events such as high school graduation. 

Indicators capture students’ academic performance, behavior, and engagement. They are 
based on readily available data such as course performance, credits earned, or attendance 
(e.g., missing 10 percent or 20 percent of instructional time) and are used to identify which 
students may be at risk of missing a key outcome measure. Indicators are validated through 
analysis. 

Thresholds, or cut points, are the values at which an indicator predicts that a student is 
likely or not likely to meet the predefined outcome measure. For example, research suggests 
that a student who misses 10 percent or more of school days as early as the first 20 days of 
ninth grade is at risk of dropping out of high school (Allensworth & Easton, 2007). 
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Any early warning system requires a comprehensive approach to identifying, diagnosing, and 
providing support to at-risk students. AIR developed an early warning system implementation 
guide that includes a seven-step implementation cycle (see Exhibit 1). This cycle draws from 
research on the use of early warning system indicators in schools (e.g., Berhardt, 2004; Love, 
2000 ). 

The process was named the Early Warning Intervention and Monitoring System (EWIMS) to 
reflect the need to move from identification to action. Practitioners use EWIMS to identify 
students who are showing signs of risk, diagnose the reasons for risk, match students to 
interventions, and monitor student progress. The cycle uses a model of continuous improvement 
whereby users revisit decisions based on additional data to make midcourse corrections. 

Exhibit 1. Early Warning Intervention and Monitoring System (EWIMS) 


STEP 1 

Establish roles and 
responsibilities 


STEP 7 

Evaluate and refine 
the EWIMS process 


STEP 2 
Use the 
EWS Tool 


STEP 6 

Monitor students 


STEP 3 

Review the EWS data 


STEP 5 

Assign and provide 
interventions 


STEP 4 

Interpret the EWS 
data 


2 This EWIMS process and earlier versions of the EWS Tool were developed by AIR under the National High 
School Center funded by U.S. Department of Education Grant S283B050028, and further refined under the REL 
Midwest contract ED-IEST2-C-0004. The content of the process does not necessarily reflect the views or policies of 
the U.S. Department of Education. 
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School-Level Lessons Learned 


The actual work of identifying and providing support to students must occur at the school level. 
Development of an early warning system requires effective local leadership and broad 
engagement of other school staff. Care must be taken not to overwhelm staff and to consider the 
school’s unique context and needs. 

Empower Action Through Effective Leadership 

To implement and sustain early warning systems, schools must rely on a strong leader supported 
by a committed, school-based team with robust knowledge of the student body and available 
resources. Specific lessons learned include the following: 

Leadership for EWIMS implementation can come from any individual who is motivated 
and granted the authority to lead the effort. In our experience, EWIMS implementation does 
not have to be led by the school principal. For example, in one Ohio high school, a guidance 
counselor convinced the school and district leaders that they needed to improve graduation 
outcomes for all students, and especially for students with disabilities. After receiving approval 
from the district and school leaders, the counselor spearheaded the early warning system’s 
implementation and motivated others to participate. The team quickly became efficient and 
effective at identifying the students most at risk and providing interventions and supports. The 
data coordinator stated that she now prioritizes this work over her other duties “because it 
actually impacts kids.” 

The school-based EWIMS team should represent a broad set of viewpoints and skills. In 

one Virginia high school, the team was led by the assistant principal and consisted of the 
manager of student information, a guidance counselor, an English teacher, a mathematics 
teacher, and a community member. The team was effective because it included a variety of 
individuals who each interacted with students in different circumstances, could speak to different 
student needs, and brought unique skills to the team. Likewise, one leader said this about how his 
EWIMS team operated: 

This is not just a one man show... It’s a matter of making sure everyone recognizes what 
their roles and responsibilities are and that everyone has a shared ownership of the 
process. There needs to be someone who is in charge, but there needs to be a strong team 
surrounding them. [EWIMS implementation] can’t be done in isolation. 

Team leaders provide guidance to their EWIMS teams and ensure that they have access to 
the information and power needed to make good decisions for students. Leaders help their 
teams move away from relying solely on intuition and toward using data to ensure that all 
students have the supports they need to stay in, progress through, and graduate from high school. 
The leader also holds the team accountable by ensuring that team meetings are productive (e.g., 
meetings have an established agenda and documentation process), that decisions are carried out 
(e.g., strategies or interventions suggested by EWIMS teams are implemented), and that the 
implementation process is aligned with the academic calendar (e.g., student information and 
incoming risk indicator data are imported into the early warning system tool prior to the start of 
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the school year; students’ absences, course failures, and behavior data are imported regularly; 
and student- and school-level reports are reviewed and interpreted after each grading period). 

Beware of Overwhelming Staff 

When schools first begin using an early warning system to identify at-risk students, the number 
of students flagged can be overwhelming. (For example, in some of the lower performing 
districts and schools in Virginia and California, more than half of the students were found to be 
at risk.) In these cases, EWIMS teams can feel paralyzed by the magnitude of the problem and 
unsure about what to do next. The following strategies have been found to help schools move 
from feeling overwhelmed to taking action: 

Focusing on a subset of students can facilitate early success. Some schools decide to focus on a 
smaller group of at-risk students (e.g., students with disabilities, ninth-grade students) at first. This 
strategy allows the school-based team to dig more deeply into these students’ specific needs while 
giving the team a chance to become more familiar with the EWIMS process. In one Virginia high 
school, the EWIMS team decided to focus on English language learners (ELLs) because the school 
was being held accountable for outcomes for this group and the graduation rate for these students 
was very low. After team members reviewed the early warning system data, they identified an even 
smaller subset of ELLs who needed academic support. These students received targeted support 
and monitoring during the program’s first semester. The decision to focus narrowly in this way 
resulted in quick action and improvement for some students while the team gained valuable 
experience in the implementation of EWIMS. It is important to note that although the team focused 
on a subset of students, it continued to support other flagged students by using existing school 
supports and interventions. For example, students who were flagged due to poor attendance were 
assigned to attendance monitoring. The examination and assignment of support was expanded 
further over time as the team became accustomed to the process. 

Students with like needs can be grouped and matched to existing supports and 
interventions. This strategy to manage an overwhelming number of at-risk students requires 
teams to conduct an inventory of all existing supports and interventions. For example, one school 
early warning system team in Ohio was shocked to learn that more than 75 percent of students 
were flagged for one or more risk indicators. To accomplish its goal of providing support to 
every at-risk student, the team developed a plan to “triage” student needs. The school used early 
warning system data and other student data to group flagged students (e.g., by percentage of 
instructional time missed or by the type and number of courses failed). The school then offered 
like students similar interventions, many of which already existed in the school. Students with 
more than one flag were offered more intensive interventions. The school continued this process 
while engaging in a longer term strategy to 
restructure and develop new interventions to 
meet students’ identified needs. 

Interventions are not just prevention 
programs. Frequently, when an early warning 
system is initially implemented, district and 
school staff rely on defined programs, such as 
a dropout prevention program that focuses on 


Key Term 

Interventions include prevention programs 
for at-risk students as well as less intensive 
supports such as academic support, behavior 
management strategies, and other supports 
not typically associated with at-risk students. 
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addressing students’ ability to stay in, progress through, and graduate from high school. 

However, when many students are being flagged as at risk, narrowly defined prevention 
programs may not meet all of the needs. Some students may need targeted academic or 
behavioral supports, or even less intense interventions that are universally available within the 
school. Interventions and supports can be broadly defined to include academic tutoring, advisory 
programs, and behavior management strategies in addition to traditional dropout prevention 
programs (Dynarski et al., 2008). For example, one Virginia district-based high school team 
examined ninth-grade students and found that a large number of ELLs were at risk. As members 
of the EWIMS team began examining the students’ academic records, they quickly discovered 
that a majority of these students had failed mathematics in the sixth grade and had shown a 
precipitous decline in attendance and course performance since that time. Rather than place these 
students in a dropout prevention program, the district offered these students support in 
mathematics, starting in the middle grades and continuing into early high school, which resulted 
in an observed increase in attendance. 

Take a Whole-Student, Whole-School Approach 

Schools can mitigate many potential barriers to implementation by developing a whole-student, 
whole-school implementation strategy that is sensitive to the school’s unique capacity and 
context. Some specific lessons learned include the following: 

Engaging staff across content areas provides a broader view of students. Many schools, 
particularly at the middle and high school levels, divide or silo staff by department or grade 
level. Although this approach serves a purpose, it may limit the staffs ability to know when a 
student is showing signs of risk across more than one subject area. Examining early warning 
system data as a whole school, or with representation from a number of school staff, allows 
educators to transcend subject-area boundaries to see the whole student. For example, more than 
half of the teachers on an early warning system team at one Virginia high school reported being 
surprised to see how many students were failing in the subject areas that they taught, as well as 
other subject areas. The group quickly began to examine course failures across subjects to 
determine whether students were struggling in a particular content area or in a number of areas. 
This allowed them to 
diagnose students’ needs and 

match them with appropriate “[The early warning system] helped the school to move from 

supports. looking at individual students to thinking about how the whole 

The early warning system school operates. ” —High school counselor, Ohio 

team cannot act alone. 

Early warning system 

implementation depends on participation from educators throughout the school. An early 
warning system team can lead the effort, but the responsibility for carrying out supports for at- 
risk students must be shared by the whole school. For example, at one middle school in central 
California, the early warning system team meets regularly, reviews data, and determines 
potential strategies and interventions for each flagged student. But the team’s most important 
work is communicating the results of these meetings with the appropriate staff, so that other staff 
members know which of their students are at risk and appropriate supports can be provided to get 
these students back on track. For example, prior to referring a failing student to a mathematics 
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tutoring program, the team confers with the student’s mathematics teacher and the available tutor 
about the current situation and proposed solution. This process ensures that input is gathered 
from the adults who know the student best and that everyone understands and agrees on the 
expectations and goals for the student. 

Early warning systems should reflect the school context. Although each step of an early 
warning system (such as the seven-step EWIMS process) is important, these steps will be 
implemented differently at each school. Differences in school implementation are necessary to 
accommodate each school’s unique strengths, needs, priorities, and systems. For instance, prior 
to implementing the EWIMS process, one middle school in California had a behavioral demerit 
system that was integrated into its student information system and used as a basis for rewards 
and consequences. When the EWIMS process was first implemented, the school continued to use 
data from the student demerit system independent of its early warning system. However, staff 
quickly realized that these data could be used to support the same effort. As a result, the school 
customized the early warning system tool to include the student demerit scores as a behavioral 
indicator of risk and invited those responsible for the demerit system to participate on the 
EWIMS team. Existing and new team members reported that this reduced duplication and made 
more efficient use of staff members’ time. 

Examination of early warning system indicators may suggest schoolwide policy changes. 

For example, one high school in Ohio had originally planned to offer an academic tutoring and 
mentoring program to students flagged by the early warning system. After the EWIMS team 
examined students’ underlying reasons for being flagged, however, the team found that a large 
majority of the at-risk students had failed biology. This realization sparked further discussion 
between district staff, the high school science teachers, and the EWIMS team. Together, they 
determined that mentoring, and even tutoring, might not be the best solution. Instead, they 
examined the skills required for succeeding in biology and discovered that many ninth-grade 
students did not possess the academic foundation skills to succeed in the course. They then 
adjusted the sequence of science courses at the school so that biology was offered in the tenth 
grade and examined other ninth-grade courses to ensure that students received the skills 
necessary to succeed in tenth-grade biology. After resequencing the courses, the school observed 
improvement in student achievement and engagement. 


American Institutes for Research 
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District-Level Lessons Learned 


Although many schools can (and do) implement early warning systems without district 
leadership or support, schools possess a range of capacity, skills, and resources. Regardless of 
where they begin, all schools can benefit from help at some point in the implementation cycle. 
Districts can play an important role in improving the ease with which early warning systems are 
implemented in schools. 

Empower Schools 

The very fact of district involvement in a school’s early warning system implementation effort 
can increase its urgency. For example, improving graduation outcomes was a priority for one 
California district; its graduation rates remained stagnant despite targeted school and district 
initiatives. The district’s visible involvement within the early warning system implementation 
process empowered school teams to allocate more time and resources to supporting at-risk 
students. Similarly, in a Virginia district, the superintendent participated in meetings focused on 
planning and implementing an early warning system at a district high school. This heightened the 
school’s sense of urgency for improving the target outcome (graduation rates), motivated the 
school leader and staff to maintain their focus on early warning system implementation, and 
helped to remove implementation barriers. 

Identify and Address Districtwide Needs 

The district’s unique vantage point allows administrators and leaders to see patterns of need 
within and across schools. One California district identified a need for early warning systems 
implementation support among several of its middle grades and high schools. To address this 
need, the district facilitated a cross-district learning community in which teams from multiple 
schools gathered to participate in training and to share implementation practices. A district in 
Pennsylvania supported its schools by examining the patterns of early warning system risk 
indicators among subgroups of students and across schools. Based on this review, the district 
conducted further analysis and worked with schools to develop appropriate student supports. 

Streamline Access 
to Data 

Timely access to data is 
critical to the development of 
early warning system 
indicators and to the efficient 
diagnosis of at-risk students’ 
needs. Many districts maintain centralized student data systems; these can be a great help to the 
individuals and teams who are implementing early warning systems within schools. For example, 
a Virginia district loaded each school’s EWS Tool, to streamline the data-importing process and 
ensure that each school had access to the latest information. Districts in California wrote 
computer program scripts so that each school’s EWS Tool could be loaded from the central 
student data system with minimal burden on the school staff. Thanks to this support from the 
district, schools were able to spend more time concentrating on students and their needs. 


“I looked at [an early warning system] as a way to sort of get 
an idea of who those kids were at risk for dropping out....It 
has expanded into really looking at how schools do their 
business day-to-day. ” —District representative, Michigan 
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State-Level Lessons Learned 


Although much can be accomplished to implement early warning systems at the school and 
district levels, states also have a role to play. Selecting or validating indicators and developing 
tools and reports to facilitate implementation can be particularly daunting tasks for districts and 
schools exploring the use of early warning systems. States are uniquely positioned to help 
eliminate these barriers. 

States seeking to create a statewide early warning system typically follow three distinct steps: 

(1) identify or validate early warning system indicators, (2) customize and develop tools to 
support districts and schools, and (3) encourage and support early warning system 
implementation (see Exhibit 2). The following sections describe state-level strategies and lessons 
learned based on observations from AIR’s work with seven SEAs. 


Exhibit 2. State Process for Implementing Early Warning Systems 



Help Districts and Schools Define and Validate Indicators 

Before an early warning system is developed and implemented, stakeholders must agree about 
the intent and use of outcomes and indicators. Lessons learned in this area include the following: 

Stakeholders may lack a common understanding of early warning systems and appropriate 
roles for each level. Although the concept is quite simple, early warning systems can have 
different meanings for stakeholders depending on their frames of reference (e.g., state, district, or 
school). For example, to an SEA, developing an early warning system may refer to the process of 
validating indicators and providing this information to educators in districts and schools as an 
early warning system. However, validated predictive indicators are just one part of the system; it 
also requires communicating about the information and taking action to support at-risk students. 

SEAs can play an important role in increasing schools’ access to validated indicators. States 
can identify early warning system indicators based on prior research, validate their own early 
warning system indicators using student longitudinal data, or use some combination of the two 
methods. For example, California and Texas have relied on preexisting research to identify the 
indicators and thresholds they use. Minnesota validated its own indicators and thresholds using 
state and district data; Massachusetts and Virginia relied on a combination of a state-validated 
indicators and those identified in prior research. 
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There are advantages to both approaches. The advantage of validating state indicators is that 
SEAs can integrate data specific to the state, such as state assessments and ELL assessments. 
Validating indicators requires an investment of resources but can improve adoption by districts 
and schools because the data are meaningful to the leaders and educators. The advantage of using 
indicators based on prior research is that it requires fewer SEA resources. Exhibit 3 provides a 
synthesis of existing research on indicators and thresholds to identify students who are at risk of 
not graduating from high school. Some SEAs, such as Minnesota and Virginia, use prior research 
to introduce the concept of an early warning system to districts and schools, then validate 
indicators in a later phase of implementation. 


Exhibit 3. Research-Based Early Warning System Indicators for Risk of Not Graduating 
From High School 3 


Indicators 

Thresholds 

Middle Grades (Grades 6-8) 

High School (Grade 9) 

Attendance 

Missed 20% or more of instructional 
time 

Missed 10% or more of instructional time 

Course 

Performance 

Failed an English language arts or 
mathematics course 

Failed one or more courses (any course) 

Earned 2.0 or lower GPA (on a 4-point 
scale) 

Failed two or more semester-long core 
courses or lacked enough credits for 
promotion to the next grade 


For states that choose to validate their own indicators, stakeholders must agree on the outcome 
measure of interest (e.g., reading by the end of third grade, high school graduation, college 
completion), define how this will be measured, and ensure that student-level longitudinal data 
are available to validate indicators. For example, Massachusetts wanted to identify students as 
early as the first grade who were at risk of not being able to read by the end of third grade—a 
widely accepted milestone. Defining this milestone or outcome measure within the constraints of 
available state data was a challenge. In the end, a proficient score on the state’s third-grade 
reading assessment was determined to be the best available data, and a reasonable approximation 
for this outcome measure. The indicators were validated against this outcome measure. 

Develop Customized Supports 

SEAs that share early warning system indicators with districts and schools quickly discover that 
this is not enough. Schools and districts often require additional support, ranging from technical 
assistance with the functionality of the early warning system tool to help understanding how best 
to address the needs of at-risk students. A framework for implementation that includes guidance 
and tools developed by the state during the planning stages can provide this type of support. The 
following are some key lessons in offering implementation supports: 

State resources can provide a framework for early warning systems implementation. 

Several states, such as Massachusetts and Minnesota, have focused on providing early warning 


3 Indicators and thresholds identified are from the work of Allensworth and Easton (2005; 2007), Balfanz and 
Herzog (2005), and Neild and Balfanz (2006). 
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systems resources to districts and schools on the SEA website or in meetings. In Massachusetts, 
the SEA regularly convened a group of districts struggling to raise graduation rates and offered 
expert presentations, materials, and resources to support their efforts. Similarly, in Minnesota, 
state materials, resources, and (in some cases) coaching were provided to districts and schools 
identified as needing support to improve graduation rates. 

Tools and data reports may need to be customized. Providing a variety of early warning system 
reports that are meaningful to the school improves the likelihood of action to support students. For 
example, a group of pilot districts in Virginia wanted to do more than just flag individual at-risk 
students; they wanted to be able to share school-level summary counts of at-risk students with a 
broader audience. Sharing this school-level report with school staff, community members, and the 
board of education resulted in increased understanding of the extent of the need for support and 
intervention in their high schools and promoted greater buy-in from all stakeholder groups. 
Likewise, a group of Texas school leaders requested a student-level report specifically for their 
ELL students. Developing this custom report allowed staff to observe patterns in core course 
failures and to develop supports to meet the particular needs of this subgroup of students. 

Schools need guidance on how to use early warning system indicators. It is essential that 
identification of at-risk students does not result in simply another label (or worse, stigma), but 
instead leads to support for those identified. In some cases, states have provided early warning 
system indicators and identified at-risk students only to leave districts and schools unsure about 
next steps. Our experience suggests that there are ways to improve the likelihood that 
identification of at-risk students results in action. For example, after Texas provided districts and 
schools an early warning system tool to identify and monitor ninth-grade students, educators 
provided feedback that they could identify at-risk students but were unclear about how to use this 
knowledge to inform the assignment of interventions. The SEA brought in external experts to 
provide trainings and develop tools and guides to support districts. Similarly in Virginia, a group 
of districts participating in a state early warning system pilot continually found that the number 
of students who were identified as being at risk was overwhelming, and that school staff needed 
additional direction. The SEA collaborated with a federally funded content center to develop a 
suite of tools and guidance that incorporated feedback from a subset of districts and schools. 
These tools and guidance are now offered to districts and schools throughout Virginia. 

Use Levers to Encourage and Support Implementation 

One of the first questions SEAs often ask about early warning systems is how to start or broaden 
participation among districts and schools. In our experience, there are many levers that states can 
use to encourage implementation of a statewide early warning system (in addition to the creation of 
customized supports, discussed in the previous section). Consider the following lessons learned: 

Some states include reporting of early warning system indicators as part of a state grant or 
other program. For example, the Texas SEA required grantees of the Ninth Grade Transition 
Initiative (a summer bridge program to support at-risk students) to follow and report on students’ 
early warning system indicators through the ninth grade. 

Legislative requirements can prompt implementation. Legislative mandates aimed at holding 
districts and school accountable for improving graduation rates can provide strong motivation for 
local implementation of statewide early warning systems. In Virginia, the state board of 
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education enacted a policy that links graduation rates to a high school’s accreditation status and 
annually raises the required graduation rate—resulting in growing numbers of high schools in 
danger of losing their accreditation. The Virginia Department of Education anticipated this 
policy shift and called on AIR experts to help develop the Virginia Early Warning System Tool 
and provide EWIMS guidance through the National High School Center and Appalachia 
Regional Comprehensive Center. The tool and guidance were offered for free to all districts and 
schools in the state, and districts and high schools that were identified as being at risk of losing 
accreditation were required to report quarterly on students’ early warning system indicators. 
Similarly, the California state legislature enacted a law requiring middle schools to be held 
accountable for high school graduation rates. In response, the California Department of 
Education collaborated with AIR experts (through the National High School Center and the 
California Comprehensive Center) to develop resources and tools targeted at the middle grades, 
as well as a comprehensive support strategy for districts to pilot these efforts. 

Pilot programs provide opportunities to improve implementation and increase district 
and school interest in early warning systems. Many states first pilot early warning systems 
with small numbers of districts and schools in order to learn how they can best support 
implementation. In California, interested districts and schools responded to a general call. In 
Virginia and Massachusetts, those that showed evidence of struggling to improve graduation 
rates were invited to participate. In each case, the SEA was able to convene districts and schools 
that had placed a priority on improving high school graduation outcomes for students. Each state 
facilitated the pilot in a way that allowed the SEA to obtain feedback about implementation and 
its challenges, and to identify solutions collaboratively (between the SEA and districts/schools 
and among schools). States reported that the pilot approach allowed them to learn more about 
implementation and to hone support strategies—such as coaching for early warning system 
implementation in Virginia or the identification of evidence-based intervention strategies in 
Massachusetts—before the system was rolled out statewide. In California, the SEA started a 
second group of pilot districts and schools the following year. Similarly, the pilot in Virginia was 
so well received that it was quickly expanded to a larger number of districts and schools. 

Pilot programs also support collaborative learning among districts and schools. As with 
many other practices, schools implementing early warning systems benefit from sharing their 
experiences with other schools. Districts participating in pilot programs in Virginia and California 
reported that they valued the opportunity to share early warning system implementation practices 
and learn from one another. In Massachusetts, the cross-site learning opportunities were considered 
so valuable that the initial number of participating districts soon doubled. In California, a cohort of 
schools piloted the implementation of the Early Warning System Middle Grades Tool. Cohort 
schools received regular (monthly) professional development and sharing opportunities from the 
state, facilitated by early warning systems experts. One participating principal noted that as a result 
of the supports provided by the state, school staff have communicated with staff from other 
schools, and communities of practice have been established. The principal offered one example of 
how participation in the cohort helped his school: 

One of the things that we... [started this year] is a new intervention period, and that was 
something that came out of meeting with other schools and talking about what you do for 
your kids who are struggling. There are two schools specifically that I still talk to—that 
we e-mail back and forth—which is really, really fantastic for me because they are in 
different areas of the state and I can get more information from them, and they’re using 
the tool, which is helpful. 
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Conclusion 


Using data for the purpose of identifying students who are at risk of missing key educational 
milestones marks a paradigm shift in public education. Once used to hold states, districts, 
schools, and educators accountable for results, using data for the purposes of early warning 
indicators holds the promise of supporting educators in identifying students at risk and efficiently 
targeting resources, supports, and interventions toward meeting those needs. When part of an 
early warning system, indicators help leaders and educators come together with a common 
purpose of placing students, and students in need, at the center of their conversations. 

An early warning system is a simple concept for improving graduation outcomes for students, 
but it is also a harbinger of opportunity to identify students who are at risk of missing other 
established milestones. For example, given the wealth of data available at the state, district, and 
school levels, it is possible to use indicators to identify students who are at risk of not being able 
to read by the end of third grade—a well-established milestone. In an age when college and 
career readiness drives many decisions in public education, perhaps there is even an opportunity 
for educators to identify at an early age which students will graduate college and career ready 
and which will need additional support to meet these goals. When used as part of a system that 
offers support to improve educational outcomes for students, early warning indicators have the 
potential to transform the work of educators and improve the lives of our students. 

Resources 

Information about the following resources is available at www.earlywarningsvstems.org : 

■ Building an Early Warning System: The Implementation Path 

■ College Persistence Indicators Research Review 

■ Massachusetts K-12 Early Warning Indicator System 

■ College and Career Readiness and Success Center College Readiness Indicators 
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